There is no 2023 video available for this lesson. You can view the 2022 video instead if you'd like, but remember, it might not match the instructions.
After completing this lesson, you’ll be able to:
There is no 2023 video available for this lesson. You can view the 2022 video instead if you'd like, but remember, it might not match the instructions.
List attributes, sometimes called lists, are FME's way of allowing an attribute to store multiple values per feature. For example, rather than creating a text field named FRUITS that stores the value “Apple, Orange, Pear”, a user can create a list, which is more structured and can be broken down into constituent parts for processing:
Fruits{0}: Apple Fruits{1}: Orange Fruits{2}: Pear
In FME, list attributes are denoted using curly brackets after the list name (e.g., this list is called Fruits{}), and a number inside the curly brackets represents the element's index inside the list, e.g., Orange is element 1 in the list. This structure allows list attributes to be reorganized, exploded into individual parts, analyzed statistically, and more.
As is common practice in computer programming, list indexes in FME start at 0, not 1.
Data can be modeled in a wide variety of ways. Some of the most common methods include:
List attributes in FME can be used to deal with these different data models in a single workspace.
Flat file databases are composed of tables that do not store relationships with other tables. These databases are intentionally simple and often small. Often, the data is stored as delimited text files, but it can also use binary data.
Flat file databases are easy to read with FME using readers from formats like CSV or Text File. In this case, list attributes are not required unless you want to store data about a one-to-many relationship on features (more on that below).
Relational databases store data in tables that can be linked to each other through defined relationships. FME can process relational data simply without the use of list attributes.
Here is an example from the United States Department of Transportation. A relational data model stores separate tables defining road maintenance activities and events. These tables can be linked by a shared key, in this case, an Activity Code:
FME would read each table as a feature type, with each row becoming a feature. List attributes are not required in this case unless you want to store data about a one-to-many relationship on features (more on that below).
Object-oriented data models store data on objects, commonly as a key-value pair.
Consider a similar dataset as the one above. The same data could instead be stored using an object-oriented model, where one object is a Maintenance Report and another is a definition of a Maintenance Activity. Data is stored as instances of these objects and can be linked by the values of certain key-value pairs, in this case, the value of Activity Code:
List attributes let you store object-oriented data in FME's feature-based or table-based framework.
When incorporating Python or R code into your workspace using the PythonCaller or RCaller, you can read Python or R lists/arrays as FME list attributes after executing the code.
List attributes can also be used to represent a nested data structure within FME's feature-based paradigm.
For example, the following JSON:
[{"name": "Apple","scientificName": "Malus domestica"},{"name": "Orange","scientificName": "Citrus X sinensis"},{"name": "Pear","scientificName": "Pyrus"}]
can be stored as an FME list attribute:
Fruits{0}.name: Apple Fruits{0}.scientificName: Malus domestica Fruits{1}.name: Orange Fruits{1}.scientificName: Citrus X sinensis Fruits{2}.name: Pear Fruits{2}.scientificName: Pyrus
Working with JSON and XML in FME requires making a decision about how to extract the nested structure into a mix of FME features and list attributes.
For more on working with nested data structures, see Getting Started with JSON or Getting Started with XML.
You can inspect list attributes using the Feature Information Window. Select the feature you want to inspect. You will notice that list attributes are not included as attributes in Table View. Instead, look for the attribute in the Feature Information Window under Exposed or Unexposed Attributes. Each index and value will have its own row in the tree. You can double-click an item to inspect the full value.
⭐ New in FME 2023.1: we have improved the display of lists in the Feature Information Window. Lists are now displayed in a nested format where each item can be expanded for futher inspection. You can use the Filter field to search for specific values.
Because they contain more values than can fit in a single cell of the Table View, list attributes cannot be exposed and will not appear in Table View or, in most cases, written data. Several transformers are available to help you extract data from list attributes and use it in your workspace or written data (see the next lesson).
Because list attributes let a single attribute store multiple values per feature, they are often used to:
Express relationships
For example, listing the names of all the stores in a neighborhood in a list called Store_Names{}. FME transformers that conduct spatial analysis, like the Overlayer family of transformers, often generate list attributes like this.
"Looping" with FME
Many new FME users ask how to create loops in a workspace. While custom transformers do support looping, you can almost always accomplish the desired goal using one of two superior methods:
List attributes can be used to avoid loops by storing all the relevant values in a list and then using a list transformer like those shown here to explode, search, sort, or otherwise extract values to find the desired information. An advantage is that list attributes process much quicker than loops. Even if you do use looping in a custom transformer, it's easier to apply a custom transformer loop to elements in a list than to a series of features.
For an example of these techniques in action, check out Question of the Week: To Loop or Not to Loop from the FME Community.
List attributes can be built manually, created automatically by a transformer, read and written by some formats (e.g. XML and JSON), or "exploded" back into single-value attributes. FME has 15 transformers for list manipulation (plus more on FME Hub; see the next lesson for details) and over 80 transformers can produce list attributes. Transformers often create a list when attributes from different features are grouped into a single feature. Transformers that create list attributes usually have a "Generate List" checkbox under the "Attribute Accumulation" section of their parameters dialog:
Checking "Generate List" can have significant impacts on workspace performance, as list attributes can drastically increase the size of individual features. It's best practice to only check this box when you need it and to remove list attributes as soon as you can so they are not unnecessarily slowing down your workspace. For more performance tips, check out the Optimize Workspace Performance course.
To learn more, visit About List Attributes in the documentation.